87 research outputs found
The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Cyber-Physical Systems
This paper analyzes the simulation to reality gap in reinforcement learning
(RL) cyber-physical systems with fractional delays (i.e. delays that are
non-integer multiple of the sampling period). The consideration of fractional
delay has important implications on the nature of the cyber-physical system
considered. Systems with delays are non-Markovian, and the system state vector
needs to be extended to make the system Markovian. We show that this is not
possible when the delay is in the output, and the problem would always be
non-Markovian. Based on this analysis, a sampling scheme is proposed that
results in efficient RL training and agents that perform well in realistic
multirotor unmanned aerial vehicle simulations. We demonstrate that the
resultant agents do not produce excessive oscillations, which is not the case
with RL agents that do not consider time delay in the model.Comment: 6 pages,4 figures, Submitted to ICRA202
Static Hovering Realization for Multirotor Aerial Vehicles with Tiltable Propellers
This paper presents a theoretical study on the ability of multi-rotor aerial
vehicles (MRAVs) with tiltable propellers to achieve and sustain static
hovering at different orientations. To analyze the ability of MRAVs with
tiltable propellers to achieve static hovering, a novel linear map between the
platform's control inputs and applied forces and moments is introduced. The
relation between the introduced map and the platform's ability to hover at
different orientations is developed. Correspondingly, the conditions for MRAVs
with tiltable propellers to realize and sustain static hovering are detailed. A
numerical metric is then introduced, which reflects the ability of MRAVs to
sustain static hovering at different orientations. A subclass of MRAVs with
tiltable propellers is defined as the Critically Statically Hoverable platforms
(CSH), where CSH platforms are MRAVs that cannot sustain static hovering with
fixed propellers, but can achieve static hovering with tilting propellers.
Finally, extensive simulations are conducted to test and validate the above
findings, and to demonstrate the effect of the proposed numerical metric on the
platform's dynamics
Real-time system identification using deep learning for linear processes with application to unmanned aerial vehicles
This paper proposes a novel parametric identification approach for linear
systems using Deep Learning (DL) and the Modified Relay Feedback Test (MRFT).
The proposed methodology utilizes MRFT to reveal distinguishing frequencies
about an unknown process; which are then passed to a trained DL model to
identify the underlying process parameters. The presented approach guarantees
stability and performance in the identification and control phases
respectively, and requires few seconds of observation data to infer the dynamic
system parameters. Quadrotor Unmanned Aerial Vehicle (UAV) attitude and
altitude dynamics were used in simulation and experimentation to verify the
presented methodology. Results show the effectiveness and real-time
capabilities of the proposed approach, which outperforms the conventional
Prediction Error Method in terms of accuracy, robustness to biases,
computational efficiency and data requirements.Comment: 13 pages, 9 figures. Submitted to IEEE access. A supplementary video
for the work presented in this paper can be accessed at:
https://www.youtube.com/watch?v=dz3WTFU7W7c. This version includes minor
style edits for appendix and reference
A stacked LSTM based approach for reducing semantic pose estimation error
© 1963-2012 IEEE. Achieving high estimation accuracy is significant for semantic simultaneous localization and mapping (SLAM) tasks. Yet, the estimation process is vulnerable to several sources of error, including limitations of the instruments used to perceive the environment, shortcomings of the employed algorithm, environmental conditions, or other unpredictable noise. In this article, a novel stacked long short-term memory (LSTM)-based error reduction approach is developed to enhance the accuracy of semantic SLAM in presence of such error sources. Training and testing data sets were constructed through simulated and real-time experiments. The effectiveness of the proposed approach was demonstrated by its ability to capture and reduce semantic SLAM estimation errors in training and testing data sets. Quantitative performance measurement was carried out using the absolute trajectory error (ATE) metric. The proposed approach was compared with vanilla and bidirectional LSTM networks, shallow and deep neural networks, and support vector machines. The proposed approach outperforms all other structures and was able to significantly improve the accuracy of semantic SLAM. To further verify the applicability of the proposed approach, it was tested on real-time sequences from the TUM RGB-D data set, where it was able to improve the estimated trajectories
Fuzzy Ensembles of Reinforcement Learning Policies for Robotic Systems with Varied Parameters
Reinforcement Learning (RL) is an emerging approach to control many dynamical
systems for which classical control approaches are not applicable or
insufficient. However, the resultant policies may not generalize to variations
in the parameters that the system may exhibit. This paper presents a powerful
yet simple algorithm in which collaboration is facilitated between RL agents
that are trained independently to perform the same task but with different
system parameters. The independency among agents allows the exploitation of
multi-core processing to perform parallel training. Two examples are provided
to demonstrate the effectiveness of the proposed technique. The main
demonstration is performed on a quadrotor with slung load tracking problem in a
real-time experimental setup. It is shown that integrating the developed
algorithm outperforms individual policies by reducing the RMSE tracking error.
The robustness of the ensemble is also verified against wind disturbance.Comment: arXiv admin note: text overlap with arXiv:2311.0501
Neuromorphic event-based slip detection and suppression in robotic grasping and manipulation
Slip detection is essential for robots to make robust grasping and fine
manipulation. In this paper, a novel dynamic vision-based finger system for
slip detection and suppression is proposed. We also present a baseline and
feature based approach to detect object slips under illumination and vibration
uncertainty. A threshold method is devised to autonomously sample noise in
real-time to improve slip detection. Moreover, a fuzzy based suppression
strategy using incipient slip feedback is proposed for regulating the grip
force. A comprehensive experimental study of our proposed approaches under
uncertainty and system for high-performance precision manipulation are
presented. We also propose a slip metric to evaluate such performance
quantitatively. Results indicate that the system can effectively detect
incipient slip events at a sampling rate of 2kHz () and
suppress them before a gross slip occurs. The event-based approach holds
promises to high precision manipulation task requirement in industrial
manufacturing and household services.Comment: 18 pages, 14 figure
Autonomous wireless self-charging for multi-rotor unmanned aerial vehicles
Rotary-wing unmanned aerial vehicles (UAVs) have the ability to operate in confined spaces and to hover over point of interest, but they have limited flight time and endurance. Conventional contact-based charging system for UAVs has been used, but it requires high landing accuracy for proper docking. Instead of the conventional system, autonomous wireless battery charging system for UAVs in outdoor conditions is proposed in this paper. UAVs can be wirelessly charged using the proposed charging system, regardless of yaw angle between UAVs and wireless charging pad, which can further reduce their control complexity for autonomous landing. The increased overall mission time eventually relaxes the limitations on payload and flight time. In this paper, a cost effective automatic recharging solution for UAVs in outdoor environments is proposed using wireless power transfer (WPT). This research proposes a global positioning system (GPS) and vision-based closed-loop target detection and a tracking system for precise landing of quadcopters in outdoor environments. The system uses the onboard camera to detect the shape, color and position of the defined target in image frame. Based on the offset of the target from the center of the image frame, control commands are generated to track and maintain the center position. Commercially available AR.Drone. was used to demonstrate the proposed concept which is equppied with bottom camera and GPS. Experiments and analyses showed good performance, and about 75% average WPT efficiency was achieved in this research
- …